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THE DISTRIBUTION OF THE TEST STATISTIC 
USED IN THE NEWMAN-KEULS MULTIPLE COMPARISON, TECHNIQUE 

James T!. Carlson, Charles E. Stegman, and Wayne. R. Applebaum 
University of Pittsburgh 

* 

1* Introduction 

f 

Tti4 purpose 6f this study was to investigate the distribution of the 

» * * 
test statistic used in the Tukey, Newraan-Keuls and Duncan methods of Per- 
is 

forming post-hoc multiple comparisons following a significant analysis of 
variance F-test. There is a problem in this distribution that is not 

4 

directly discussed in the literature we have surveyed but is mentioned in 
a personal communication from Rupert G. Miller (1971), and is alluded to 
by Spj^tvoll(1974) . Because different Studenti2ed range distributions are 
used directly in determining critical values for the Newman-Keuls method 
our discussion will center around this particular method. 

« 

\ 

2. The Newman -Keuls Technique 

Following a significant analysis of variance F-test, among the many 
available post hoc multiple comparison techniques is one first mentioned 
by "Student" (Newman, *1939) , developed by Newman 1 (1939) and rediscovered 
by Keuls (1952). This procedure is now known as the Newman -Keuls Method. 

Given the set of k population means, y^, v^* •••• \ > estimated 
by the means of k independent random sample, Y 1f Y 01 Y, , the first 

step in the Newman -Keuls procedure is to rank these sample means. Denoting 

' th — / 

the j smallest mean as so that the smallest is Y^ and the largest; 

^ 8 *00 • ^ ordered set of . sample means is *(2) f ^(k) * 
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The second step is to compute the , following ratio: ,Q,, /1N < 



where n is the size of each of the k samples and !IS e is the error mean /n 
square from the- analysis of variance (note that the technique assumes that 
the k samples are all of the same size, n) . This ratio is then compared 
t0 q k,v,l-jl » the i 00 * 1-0 ) percentile of ,the Studentized range distribution 
with parameters k and v, the number of degrees of freedom associated ,with ~ 
the error sum of squares in the analysis of variance. 



If 



k Q (k)-(l) > q k,V,lHl 



HI 



the hypothesis of no difference between the means of groups (k) and (1) is 
rejected and the researcher proceeds to test other pairs of means. If, 
'on the other hand, 



k g (k)-(l) < q k,v,l-it 



[2] 



the researcher concludes that the two most extreme group means are not .dif- 
ferent. Since the two most extreme group rtean^ are judged not to be 
different, the researcher also concludes that all other, less extreme, 
pairs are not different. In this case, no further comparisons are made. 

If the inequality in [1] is true, the next comparisons to be made are ' 
those represented in the inequalities. 



k v (k-l)-(l) 



Y (k-1) " Y (l) 



J MS fi / n 



* q k-l,v, l»a 



[3] 



arfd 



k Q (k)-<2) " 



Y (k) ~ Y (2) 
7 "So / n 



> q k-l,v,l-« 



[4] 
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If [3] is true the researcher 'proceeds to examine Y ( k . 2 )^N Y (l) ' but 



{3] is not true theh the two means y 



(k-2) 



and are judged not to be 



different. Similarly, if [4] is true - Y (2) and Y (fc) - Y (3) are 

examined, but if [4] is not true the analogous population means are judged 
not to be different. If we construct a table of differences between pairs 

of ordered means, as illustrated in Table 1, it can be seen that this 

v 

procedure results in the following rules: (a) we start by examining the 

j 

difference in the upper right corner and if it is significant we proceed 
to examine the next difference in the same rou and the next difference in 
the Sana column, (b) when a nonsignificant difference is found, no further 
differences in the ^ame row and the same column are examined. 



i Table 1 

Mean Difference Table for NewmanrKeuls Technique 



7(2)." Y (D. 


Y (3) " Y (l) 


• • . . 


Y (k-1) ' Y (l) 


Y - Y ' 

X (k) Y (l) 


' N 


Y (3)-" Y (2) 




Y (k-1) - Y <2) 


Y (k) ~ Y (2) 


_ 




/ 


* • 


• 
« 
« 








Y (k-1) ~ Y (k-2> 


Y (k) ' Y (k-2) 










Y (k) "X(k-l)j 



) 
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3. The Problem . * 

The ,jise of the q. distribution for the distribution of the test 
statistic, jjQ^wj.')'* a8 a test of 

is valid when the usual ANOVA ' assumptions are met (Spj^tvoll, 19 74). That 
is, given normality, independence,, homogeneity 6f variance, and the fact 

•that the hypothesis is^true then the variable fa (in formula 1) 

s~ * ' - > 

'is distributed as the Studentized range with k and v degrees of frfeedora. 

As Spjtftvoll notes, if we reject H , using the decision rule in [1] 

^ / ol * 

then we '"conclude" tliat y fc £ y^ In t^is case, it is tenable that it 

is population k that is different from the others or that it is population 

1\ -Therefore, w.e have two possible simplifying hypotheses: 



and 



H o2 ; ^1 a »*2 ~ '•' =* ^k-l 



H o3 S ^2.° "3 " — " 
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These correspond respectively to the cases where y^ and y^ lead to the 

rejection of H «, . • 

J ol < 

Nbw if it is true that H * is false then in H 0 and H * we have chosen 
ol o2 o3 

to remove a^pppulation mean Ceither y^. or y^) that is indeed' different froia 
the others. In this case, the removal of this extreme value should not 
effect .the order statistics of the remaining k-1 means and we 'can use the 

test statistics ^(k^i)-^ an( * k^(k)-(2) to test **02 H 03 r ? a P ect ^ ve ^y • 
Again, if assume H ^ H ^ a * e true and the ANOVA assumptions are met 
then the variables ^ and ^Q^)^^) s ^ ou ^^ ^ distributed as the 

Studentized range with k-1* atjd v degrees of freedoin. 



•'V 



However, if we made a Type I error in "concluding" that H q1 is false 

then the distributions of the variables ,Q,, 1X and ,0 /lN , 0 . are not 

kHk-l)^(l)* k (h)-(2) 

the results of k-1 independent samples. Rather they represent k-1 samples . 

remaining after discarding one extreme observation. This has the effect 

of truncating or restricting the range of values for. th£ test* statistics. 

It was hypothesized that this would reduce the actual mean and variance of 

the distributions and consequently will lead to a more ^conservative test 

of H q2 and H q ^. The same argument applies to H^^H^, etc. if we rejected 

H q2 or H q3 on the basis of fc Q (k .. lwl) or k Q (k) _( 2)< 

Of course, if we knew that H ^ were actually true th$n following the^ 
logic of the testing procedure we would not test ^ Q y etc * Howejvey, 

in practice we make the original -decision on the basis of ^Q^j.qj and 

indeed Type I" errors will be made. Sincfe t|fis is a problem for all such 

V 

comparisons, a natural question is, "How different are the true distribu— 
tions of the Newman-Keuls test. statistics from the Studentized. range dis- 
tributions that are actually used?" v 

Since the mathematical. derivations and subsequent numerical integrations 
to .determine the critical points for the actual distributions are quite 
complex, a Monte Carlo sampling study was designed to estimate how conserva- 
tive the test procedures are. The design of the study .is discussed in 
Section 5/ 



4. ' Related Literature 

Although we were unable to find any literature relating directly to 
the problem of this study thero^are^sevef£l sources that provide informa- - 
tion about the Newman -Keuls technique and its use. 

As mentioned earlier the technique examined in this study was first 
proposed by Newman (1939) and later Keuls ' (1952) , apparently unaware of 
Newman's paper, proposed the same technique. Newman, and Miller (1966) 
both indicate that the basic idea was an outgrowth of the work of •"Student," 

The most comprehensive textbook discussions of the Newman '-Keuls tech- 
nique are thpse of Miller (19^6, pp. 81-90) and Winer (1971, pp. 191-196). 
Miller presents most of the underlying theory and compares the technique 
to alternative techniques, in particular to the Duncan multiple range 
test. Winer presents the computational procedures and also presents a 
comparison of seven different multiple comparison methods, including the 
Newman -Keuls method* Other texts that. discuss the technique include those 
'of Kirk (1968) and <Ftendenhall (1968). 

Besides the above-mentioned textbook discussions there are several 

journal articles that include discussions of the Newraan-Keuls technique. 

The most comprehensive of these is the article by Spjrftvoll (1974) in 

which the bases of several procedures and comparisons^ of them are discussed. 

O'Neill and Wetherill (1971) briefly mention the Newraan-Keuls technique 

in an interesting "state of the art"- discussion, and also provide an 

extensive bibliography on multiple comparison procedures. Two studies' 

(Petrinovich & Hardyck, 1969; Carmer & Swanson,' 1973) have been conducted 

in which Monte Carlo methods were used to compare the Type I and Type II 

error rates of several multiple comparison technqiues. Although the Newman- 
's^ 

Keuls technique was investigated in both of these studies the conclusions 



were quite different. fPetrinovich and Hardyck' argue against using the' 
Newman-Keuls technqiue because they found it had a high experijaentwise 
Type I error rate under conditions other than the case wh^re all popula- 
tions have identical means. Carmer and Swanson, on the other hand reject 
the Newman-Keuls .procedure because of its Type II error rate. Games (1971) 
provides some additional discussion in a criticism of the Petrinovich and 
Hardyck article. Interestingly, Carraer'^nd Swanson generated their d^ta 
according to a randomized block design with zero block effects, and " ; 
analyzed the data according to that model. With zero'blocfc: effects the 

correct model would be the completely randomized model and, in general, 

5 

analysis according to -an incorrect model leads to^ a loss in precision. 
• The authors do not indicate why they chose the blocking clesign. / 

5; Design of the* Study 

In £rdef to investigate the sampling distribution of the test statistic 
used in the Newman-Keuls technique a total of 9 experiments were simulated, 
each with 10,000 replications.. The fiijst three experiments each had three 
groups and used samples of size 5, 10, and 15 respectively. The next three 
experiments involved four groups ^gain using sample sizes of>5, 10 and 15. 
The final thiee experiments were explicitly designed to generate empirical 
sampling distributions of the specific. Studentized range distributions that 
would be needed in some of the first six experiments. The descriptions of 

the nine experiments shown in Table 2, include the error degrees of freedom 

a 

in order to indicate in which experiments the distributions are comparable. 



V 
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Table 2 

Descriptions of the Simulated Experiments 



Experiment No. 


1 


2 


3 


4 


5 


6 


7 


8 


9 


No. of Populations 


3 


3 


3 


4 


4 


4 


2 


2 


3' 


Size of each Sample 


5 


10 


15 


. 5 


10 


15 


7 


22 


13 


Error Degrees of 


' 12 ' 


27 


42 


16 


36 . 


56 


' 12 


42 


36 



Freedom 



For each of the possible contrasts in each experiment .the frequency 
with which the value of the test statistic fell in each of seven intervals 
was tabulated. The seven intervals were defined by using percentiles of 
the particular Studentized range distribution that would be used for the 
Newnan-Keuls technique oij each contrast. The percentiles (.900, .950, 
.975, .990, .995, .999) were taken from Harfcer's (1960) tables or calculated 
by linear harmonic interpolation of values from those tables as suggested 
by Harter. The upper tail percentiles wete chosen because they are the 
critical areas of the distribution used by the Newraan-Keuls technique. 
Thus, using Q to represent the value of the test statistic and P* to 
represent the j percentile of the appropriate Studentized yange distribu- 
tion-, the seven intervals that were uped were: Q £Pgg» Pg Q < Q £^955 

P 95 < ^ - P 97.5 ; P 97.5 < ^ - P 99' P 99 < ^ — P 99.5' P 99.5 < * i P 99.9 ; «* ' • 

P 99.9 <Q ' . 

Estimates of the mean, variance, skewness and kurtosis were also cal- 
culated for each distribution, using the formulas given by Bennett and 
Franklin (1954, pp. 81-82). Using the method described by Newman (1939) 
expected values of each theoretical distribution were computed and com- 
parisons were made between these expectations and the means of the distribu- 
tions that >were generated. 

* 10 
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It was anticipated that the distributions for the test statistics 
would show less variability than the theoretical distributions for contrasts 
Involving pairs of means other than the\wo most extreme for a particular 
experiment . 



6. Generation Techniques m # 

Values of the dependent variable were pseudo-random standard normal 
deviates generated by using the log-and-trig transformation method (Box & 
Muller, ,1958) on pseudo-random uniform yalues generated by the multiplicative 
congruential method". The multiplier for the congruential method was 131075, 
which was selected because it has been demonstrated to result in variates \ 
having desirable characteristics (Pingel, 1975; MacUren & Harsaglla, 1965 ). 
All computations were done on a PDP10 computer at the University of Pittsburgh 

Computer Center. This machine stores an integer in a location having 35 

v 35 
bits plus sign, and thus residues mod (2 -1) were usjed by allowing integer 

t 

multiplications to overflow. For more details of this technique the 
interested reader is referred to Newnan and Odell (4£71) ot Hammers ley and 
Hfcndscomb (1964). * " 

In order to check the^computer programs, samples of the pseudo-random 



! 
1 

uniform and normal variates were generated and unb^sed estimates of the 

. . ' , - i . v * 

means and variances ^computed. -A satonle of 10,000 uniform values had 



a mean of .49945 and variance of .082^8 as, compared to the theoretical values 
of .5 and .08333, respectively. Thre^ samples, each of 10,000 normal values 

were generated and had means of -.012^4, -.00344, and .0105Z and variances 

i 

of 1.00182, .99960 and .99350 as compared to the theoretical values of zero 

i 

and one. Finally, using the final version of the computer "program, a total 
sample of 3000 normal values was generated and the mean and variance estimates 
were .00858 and .99565 respectively. . 1 ' 
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71 Results . 

t ' - * 

Tablea 3 and 4 8how " the frequencies with which values of the Newman- 
*• Keul8 te8t statistic were distributed across the seven intervals defined 

•j>y using percentiles of the theoretical Studentized range distribution. 

For' rows 1, 4, 5, 8, and 11 the Studentized range distribution is 
theoretically correct. As can be seen by examining these rows the 
observed frequencies correspond closely to the expected frequencies. 
Row 1 corresponds £o the distribution of 3 Q (3) _ (1) and the Studentized 
range distribution is the appropriate distribution. As outlined before; 
if a Type I error is made in this test then we would proceed to incorrectly 
test 3 Q (2 )..( 1 ) and 3 Q <3)_(2) * , tta distribution of these statistics do not 
theoretically have Studentized. range distributions. The observed fre- 
quencies in rows 2 and 3 clearly indicate that' these two tests -deviate 
markedly from the expected frequencies. The deviations are in the pre- 
j dieted direction, that is, the tests ^uld be conservative. For comparison 

% • purposes we generated the data in row 4 which has the same degrees of freedom 
as rows 2 and 3. However, this distribution is theoretically c6rrect because . 
it involves the contrast between the two extreme means in a two population ' 
case. Recall from Table 2 thaf in order to get comparable degrees of free- 
dom the sample size in experiment 7 were equal to seven. Further examina- 
tion df Table 3 for the cases where h-10 and n-l^supplies further evidence 
for the conclusion that if a Type I error is mad&for'the first contrast 
tested by the Newman-Keuls technique then the significance .level, for. the 
next two tests is much smaller than the nominal level the researcher may * 
think he Is using when he performs the tests. That is, when ( k-3 the sample 
sizes, and hence the number of error degrees of freedom, appear to make little 
difference i n the conservativeness of the tests. f 

ERJC , 12 - 
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Similar conclusions are reached after examining rows (1,2,3; 7,8,9,10; 
14,15,16) of Table 4. *These cases represent the situations where k°4 and ' 
n-5, 10, and 15. It is easy to see again that when a Type I error is 
made in the first test then the resultant tests based on.- ^ and 

4 Q (3)-tt) 8re cons6rvat f ve - Next, consider rdws (4,5,6; 11,12,13; 1?,18>,19) 
of fable 4. Examination of these rgwtf^shows, as might be expected, that ' 

• the distribution of tjie.^test statistic is even more drastically affected 
when tfro or three Type I errors are -made. In this case we are conducting 
test 8 of means receiving adjacent ranks for four-population experiment's. 

In all cases considered here, the Newman-Keuls technique becomes yery 
conservative with respect to Type I errors once one or more Type I, errors 
have been committed. This finding is in agreement with the. author's 
a priori hypothesis that the distribution of the test statistic would^ have 
a smaller mean and variance^under these conditions than when no Type I * 
•error was made. 

-As a further illustration, of the departure of tfte actual test statistic 

• distribution from that used to obtain the critical values we present, in 
Figures 1, 2, and 3, plots of the empirically generated ^distributions for- 

2^(2)-(l)' 3^(2)-(l) mA 3^(3)-(2)' 111686 three distributions were chosen 
because it was possible td design, for example /Figure 1) -a, two-population 
experiment with^ 12 degrees of freedom, for which q 2 ^ i8 the theoretically 
correct test statistic distribution, and a three-population experiment which 
would result in the use of q 2>12 after hfl vlng made a Type I error. 
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Figure 1. Empirical Distributions 
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Figure 2, Empirical Distributions 
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Comparisons of the actual means of the generated distributions with the^ 
expected values of the Studentized range distributions used in the Newman- * 
Keuls , technique are presented in Table 5. This Table has fcour sections. 
The first section contains the comparisons for r the distributions that are 
theoretically*correct (no Type I errors, previously made). The other sections 
contain, the comparisons for cases in^which one, two" and three Type I errors 
have been previously .made and the expectations in "these sections are those 
for the specific Studentized range .distributions used in the Newman-Keuls 
technique. ^ 
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. Table 5- 

Comparison of Empirical Heans and Thpir Expectations 



Statistic 



No. of Distri- 
Previous bution 
•Type I Used 
Errors 



lie an 



( Expectation 



Difference 
Mean-E(feean) 



-2 Q (2)-(1) 


12 


0 


- q 2,.12. 


1.2116 


1 ^ 205*6 


.0060 


5 2. Q (2)"-(1) 


'«2 


• 0 


q 2,12 ^ 


1..1665 


1.1490. 


.0175 


3?(3)-(l) 


12 


0 




1.8099. 


1.8084 • 


.0015 


3 Q (3)-<1> 


27. 


0 


q 3,27 


1.7416 


1.7415 


.0001 


3 Q (3)-(i) 

* > 


36 


0 


q 3,36 


1.7154 


• 1.7289 . 


-.0135 - 


3 Q (3)-(1) 


42 


0 


q 3,42 


1.7191 


1.7236 


-.0045 


4 Q (4)-(f) 


16 

f 


0 


q 4,16 


2.1720 


2.1620 


.0100 . 


4 Q (4)-<1) 


" . 36 


0 


q 4,36 


2.0897 


2.1029 




4 Q <4)-<D 


56 


- o • 

* 


q 4,56 


. 2.0767 


2.0868 


-.0101 


ri Q (2)'w(l) 


12 


, 1 


q 2,12 


.9079, 


1.2056 


-.2977' 


3 Q (3)-<2>- 


12 




q 2,12' ' 


.9019 


1.2056 


-.3037 & 

* 


3 Q (2)-(1) 


27- 


1 


q 2,27 


' ' .8722 


1.1609 


-.2887 


3 Q (3)-(2) 


27 


1 


q 2,27 


.8694 


1.1009 


-.2915 


3 Q (2)-(1) 


36 


1 . 


q 2,36 


.8642 


'•"1.1526 


-.2894 


3 Q (3)-(2) 


36 


1 


q 2,36 * 


.8512 


.. 1.1526 


-.3014 


3 Q (2)-(1) 


42 


1 


• q 2,42 


.8501 


1.1490 

* 


-.2989 


3^(3) -(2) 






q 2,42 


.8690 


1.1490 


-.2800 


4 Q (3)-<1) 


16 




q 3,16 


1.4082 


1.7774 . 


-.3692 ' 


4 Q (4)-(2) 


16 




q 3,16 


1.3932 


1.7774 . 


-.3842 


"4 Q (3)-(1) 


36 


<* 


q 3,36 


1.3440 


1.7289 


-.3849 


A Q (L4)-<2) 


36 




q 3,36 • 


1.J529 


1.7289 


-.3760 


4 Q (3)'-(1) 


56 




q 3,56 


" 1.3443 


* 1.7157 


-.3714 


4 Q <4)-<2) 


56 




%,56 


' 1.3396 


1.7157 


-.3761- 
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Table 5 


(cont.) . 




i 


■ 




No. of . 


Distri- 






^ 


Statistic 


V 


Previous 
Type I 
Errors 


bution 
Used 

1 


llean * 

v 

> 


Expectation 


Difference 
Mean -E (Mean) 




* 16 


2 


Qo t s 


.7788 


1.1850 


-.4062 




16 


2 


Qo i r 

/2,16 


.7638 


* 1.1850 

• 


-.4212 


4 Q (2)-(1) 


36 


2 


q 2 36 

9 JO 


.7368 


1.1526 


-.4158 




36 

— ' 


2 


Qo ■ 

n 2,36 


.7457 


1.1526 


-.4069 * 




56 


2 


Qo 

n 2,56 


.7371 


1.1438 


.-.4067 


4 y (4)-(3) 


56 


2 


q 2,56 




X . X** JO 


— A 1 1 A * 


4 Q (3)-(2) 


16 


3 


q 2,16 


.629*L 


1.1850 


-.5555 


4 Q (3)-<2) 


36 


3 


, q 2,36 v 


.6073 / 


1.1526 


-.5453 


4 Q (3)-(2) 


56 


3 


q 2,56 


76072 


1.1438 


-.5366 



[I 

Values in the first section ; of Table 5 show that the means of the generated 
distributions are very close to their expectations, the largest deviation from 
expectation for the 9 distributions being only .0175. After one Type I error 
has l^eiTmade, howevej?, the mean of the empirical distribution -differs from 
the expectation of the distribution used'' in the Newman-Keuls procedure bv 
amounts ranging from .2800 to .3849 for the 14 different distributions. 
Similarity, the mean of the empirical distributions for the case of two previous 
Type I errors deviate from the expectations of the distributions actually used 
by amounts, varying from # 4p62 to .4212. for six, distributions, and the similar 
deviations for- three distributions following three Type I erro.rs range from 
.5366 to .5555. . ' 
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These data indicate* as we had predicted, that the means of the dis- 
tributions of the Newman-Keul^ test statistic definitely do decrease following 
a Type I errdr on a previous test. 

As a final basis for comparison of the distribution^ generated in this 
study we present, in Table 6, estimates of the means, variances, and indices 
of skewnes? (gamma-one) and kurtosis (gamma-two). For convenience of reading 
this Table presents the particular contrasts in the same order as used in 
Tables 3 and 4. . 
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Table 6 

Estimates of Parameters of the Distributions 



Statistic 


V 


no . or 
Previous 




Estimates of 


Parameters 






Errors 


M*»,nri 

1 ICOU 


V*jyH on r»o 
v allautc 


oKeWucSo 

V 


KUXXOSIS 


3 Q (3)-(1) 


12 


0 


1.8099 


1.1353 


1.2098 


2.8572 


3 Q (3)-(2) 


12 


1 


:9019 


.6025 


• 1.5351 


3.3147 


3 Q (2)-(1) 


12 


1 


.9079 


.6077 
.9314 


1.5934 


4 .0802 


2 Q (2)-(1) . 


12 


0 


1.2116 


1.2794 


2.2504 


3 Q (3)-(1) 


27 


0 


1.7416 


.9274 


.9130 


1.2369 


3 Q (3)-(2) 


27 


1 


.8694 


.5137 


1.3637 


2.4987 


3 Q (2)-(1) 


27 


1 


.8722 


.5123 


1.2731 


1.9543 


3 Q ( 3 )-(i) 


42 


0 


1.7191 


.8723 


.7098 


.499Q 


3 Q (3)-(2) 


42 


1 


.8690 


.4971 


1.2010 


1.5185 


3 Q (2)-(1) 


42 


• 1 


.8501 


.4831 


1.2271 


1.6903 


2 Q (2) -(1) 


42 


0 

1 


1.1665 


.7825 " 


1.0681 


1.1629 


3 Q (3)-(2) 


36 . 


1 


.8512 


.4963 

• 


1.2901 


" 2.0159 


3 Q (2)-(1) 


36 


1 


.8642 


.4983 


• 1.2745 


2.0331 


4 Q (4)-(1)' 


16 


0 


.2.1720 


1.0523 


.8727 


1.2383 


4 Q (4)-(2* 


16 


1 


1.3932 


.6794 


.1.0277 


1.6104 


* Q (3)-(1) 


16 


1 


1.4082 


.6784 


1.0156 


/" 1.5548 


; 4 Q (4)-(3)' 


16 


2 


.7638 


' .4262 ■ 


1.4196 


2.5623 


4 Q (2)-(1) 


16 


2 


.7788 


.4306 


L.3750 


2.4693 


4 Q (3)-(2) 


16 


3 


.6295 


.2991 


*-1.5^81 


3.4331 
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Previous 



Statistic 


v 


Type I \. 
Errprs 


Moon 






IV LSI. bUSXD 


4 Q (A)-(D 


36 


0 


2.0897 


.8750 


.6720 


" .5417, . 


4 Q <4)-(2) 


36 


1 


1~3529 


.5896 


..8581 


. . 8951 


4 Q (3)-(1) 


36 


i 


1.3440 


.5820 

i 


.8139 


.7356 . * 


•3 Q (3)-(i; 


) 36 


o ' 


1.7154 


.8963 - 


.8356 


' .8442 


4 Q (4)-(3) 


36 


2 


.7457 


.3968' 


1.3975 


2.6537 . 


4 Q (2)-(1)' 


36* 


2 


.7368 


.3877 


1.2802" 


1.8960 


4 Q (3)-(2) 


36 


3 • 


.6073 


.2590 


1.2802 


4 T.8353 


4 Q (4)-(D 


56 


X) 


2.0767 


.8397 


.6263 


.4946- 


4 Q (4)-(2) 


56 


1 


1.3396 


..5633 


• .8328 


.8877 


4 Q (3)-(1) 


56 


1 


1.3443 


.5804 


.7862 


'.5126 


4 Q (4)-(3) 


56 


2 


.7324 


.3677 


1.2768 


1.9909 


4 Q (2)-(1) 


56 


2 


.7371 


.3881 ' 


1. 3349 


2.0971 


4 Q (3)-(2) 


56 


3 


.6072 


.2649 


1.3264 ^ 


2.1171 
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Examining first the estimates in Table 6 for the cases of theoretically 
correct distributions we can see several tendencies. For these cases the dis 
tributions are positively skewed and leptokurtic. As is to'be expected the 
means and variances for the same number of groups (k^ decrease as the degrees 
of freedom (v) increase, and increase a& k increases when v remains constant. 
Ttiere is a definite tendency for th^ degrees of skewness and kurtosis to 
•decrease as v increases for the same k and also a tendency for these indices 
to decrease as'k increases ^ith v helti constant. 

Next we examine those cases' in which one Type I error has been committed^ 
previous to the test of a mean difference, and for which we were able to 

compute estimates of the parameters of the distribution that Would be used in 

\ 

actually applying the Newman -Keuls technique* For example, x*e compare the 
estimates of parameters of 3 Q( 2 )^(i) 8X1(1 3 Q (3)-(^) witl \ tho8e for 2 Q (2)-(1) 
for the same number of degrees of freedom* On jnaking these comparisons* we 
seq that, as we had, originally hypothesized, the means v and variances are 
smaller than tne values from the" distribution that would be used in applying 
the Newm^n-Keuls procedure. There is also a tendency for the actual dis- 
tributions generated under the condition that one Type I error has been made 
to be more skewed and more leptokurtic than the theoretical distributions. 
In all cases the tendency is for all estimates. to decrease in value as v in- 
creases for the same k, for the mean and variance of these distributions to 
decrease as k increases for the same v; and for the indices of skewness and 
kurtosis to decrease as k increases for the same v. 

Further examination of the data reveals that, when more than one Type I 
error has been committed the mean and variance decrease even further but the 
indices of skewness and kurtosis appear not to be greatly affected. 
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4 * * 

* 8. Discussion # 

As had been expected by the authors prior to generating the data reported 
in this paper, the distribution of.,the Hewman-Keuls test statistic changes 
shape after a Type I error is committed. Also, ah expected this change' 

* 6 

reaultS' in an actual .Jype I error rate that is smaller than the significance 

'. V . 

level chosen by the researcher. This means, of course, that the Newman-Keuls 

» >i * * 

procedure becomes rather conservative when one Type I error has been ,tnade, 

and even more conservative following additional Type I errors. Thus the 

.chance of' making two or moire Type I errors is lower than would be expected 

if the distributions were not affected by the results of the previous 

significance tests. . 

It is difficult to attempt to telate our findings to those of Petrinovich 

and Hardyck (1969) and Gartner and Swanson (1973) because there no reason 

for them to estimate separately the error rates of second and third comparisons 

*■ « 

i 

after making errors on thfe initial comparison.* Our results do, of course, 
relate to the fact that Petrinovich and Hardyck found the .experimentwise 
Type I error rate for the case of all populations having identical means to A 
be better than that for mixtures of populations. 

Spj^tvoll (1974) in his .Section on the justification of the Newman-Keuls 
technique makes the statement, "At step 2 we assume (italics ours) that the 
conclusion reached at step 1 Is correct [p. 100]." Our findings point out the 
importance of this assumption. '*'.'* 
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